[KVCache] Support environment variable overrides for AttentionStore c… by jackyYang6 · Pull Request #7455 · PaddlePaddle/FastDeploy

jackyYang6 · 2026-04-17T03:05:16Z

Motivation

AttentionStore's namespace, pod_name, and model_version are currently only configurable via constructor arguments. In K8s deployments, these values are typically derived from Pod environment variables (e.g., AS_NAMESPACE, AS_POD_NAME, AS_MODEL_VERSION). Without environment variable overrides, deployment configuration is inflexible.

Modifications

Modified file: fastdeploy/cache_manager/transfer_factory/mooncake_store/attention_store.py

Added environment variable overrides for AttentionStoreConfig fields before SDK initialization in AttentionStore.__init__:

self.config.namespace = os.getenv("AS_NAMESPACE", self.config.namespace)
self.config.pod_name = os.getenv("AS_POD_NAME", self.config.pod_name)
self.config.model_version = os.getenv("AS_MODEL_VERSION", self.config.model_version)

When environment variables are not set, the original constructor defaults are preserved (no behavior change).
Environment variables take precedence over constructor arguments when set.

Usage or Command

Set the corresponding environment variables before launching the service:

export AS_NAMESPACE=my-model
export AS_POD_NAME=my-pod-0
export AS_MODEL_VERSION=v1

Accuracy Tests

Not applicable — this change does not affect model outputs.

Checklist

Add at least a tag in the PR title.
Format your code, run pre-commit before commit.
Add unit tests. Please write the reason in this PR if no unit tests.
Provide accuracy results.
If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

paddle-bot · 2026-04-17T03:05:24Z

Thanks for your contribution!

codecov-commenter · 2026-04-17T04:36:27Z

Codecov Report

❌ Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@e952720). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...transfer_factory/mooncake_store/attention_store.py	25.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #7455   +/-   ##
==========================================
  Coverage           ?   73.47%           
==========================================
  Files              ?      398           
  Lines              ?    54979           
  Branches           ?     8613           
==========================================
  Hits               ?    40398           
  Misses             ?    11873           
  Partials           ?     2708

Flag	Coverage Δ
GPU	`73.47% <25.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

hong19860320 · 2026-04-17T07:22:23Z


        try:
+            self.config.namespace = os.getenv("MODEL_ID", self.config.namespace)
+            self.config.pod_name = os.getenv("FD_POD_NAME", self.config.pod_name)


这个Pod Name 需要和上报给IC 和 IM 的pod name 对齐吗？

已确认不依赖此作为索引

hong19860320

LGTM

…onfig

PaddlePaddle-bot

🤖 AI Code Review | 2026-04-20 18:01:43

📋 Review 摘要

PR 概述：为 AttentionStore 的 namespace、pod_name、model_version 新增环境变量覆盖能力，并根据 ENABLE_EP_DP_IN_FD 拼接 splitwise_role 和 dp_id 到 pod_name。
变更范围：cache_manager/ — AttentionStore 配置初始化
影响面 Tag：KVCache

📝 PR 规范检查

PR 描述中的代码示例与实际 diff 不一致：描述仅展示了 AS_NAMESPACE、AS_POD_NAME、AS_MODEL_VERSION 三个环境变量覆盖，但实际代码还新增了 ENABLE_EP_DP_IN_FD 控制的 pod_name 拼接逻辑以及 splitwise_role 字段，建议同步更新 PR 描述。

问题

级别	文件	概述
🔴 Bug	`attention_store.py:70`	`int(os.getenv("ENABLE_EP_DP_IN_FD", ...))` 对非数字字符串会抛 ValueError，导致初始化失败

总体评价

环境变量覆盖的设计思路合理，但 ENABLE_EP_DP_IN_FD 的解析方式存在健壮性问题，需要修复。

PaddlePaddle-bot · 2026-04-20T10:03:24Z

+            self.config.namespace = os.getenv("AS_NAMESPACE", self.config.namespace)
+            self.config.pod_name = os.getenv("AS_POD_NAME", self.config.pod_name)
+            if int(os.getenv("ENABLE_EP_DP_IN_FD", "1")):
+                self.config.pod_name = self.config.pod_name + "_" + self.config.splitwise_role + "_" + str(self.config.dp_id)


🔴 Bug int(os.getenv("ENABLE_EP_DP_IN_FD", "1")) 当环境变量被设置为非数字字符串（如 "true"、"false"、"enabled"）时，int() 会抛出 ValueError。

由于此处位于 try ... except Exception 块内，异常会被捕获并导致整个 AttentionStoreSDK 初始化失败，且错误信息会令人困惑（不会提示是环境变量格式问题）。

建议使用更健壮的解析方式：

def _str_to_bool(val: str) -> bool: return val.strip().lower() not in ("0", "false", "no", "off", "") if _str_to_bool(os.getenv("ENABLE_EP_DP_IN_FD", "1")):

或者至少增加异常处理：

try: enable_ep_dp = int(os.getenv("ENABLE_EP_DP_IN_FD", "1")) except ValueError: enable_ep_dp = 1 logger.warning("ENABLE_EP_DP_IN_FD 环境变量值无效，使用默认值 1") if enable_ep_dp:

jackyYang6 had a problem deploying to Metax_ci April 17, 2026 03:05 — with GitHub Actions Failure

This comment was marked as outdated.

Sign in to view

jackyYang6 force-pushed the kvcache/as-config branch from 9fbff4a to 8407b8a Compare April 17, 2026 05:21

jackyYang6 had a problem deploying to Metax_ci April 17, 2026 05:21 — with GitHub Actions Failure

hong19860320 reviewed Apr 17, 2026

View reviewed changes

hong19860320 previously approved these changes Apr 17, 2026

View reviewed changes

jackyYang6 dismissed hong19860320’s stale review via 2d24f4b April 20, 2026 07:18

jackyYang6 force-pushed the kvcache/as-config branch from 8407b8a to 2d24f4b Compare April 20, 2026 07:18

jackyYang6 had a problem deploying to Metax_ci April 20, 2026 07:18 — with GitHub Actions Failure

This comment was marked as outdated.

Sign in to view

[KVCache] Support environment variable overrides for AttentionStore c…

7755c6b

…onfig

jackyYang6 force-pushed the kvcache/as-config branch from 2d24f4b to 7755c6b Compare April 20, 2026 09:51

jackyYang6 had a problem deploying to Metax_ci April 20, 2026 09:51 — with GitHub Actions Failure

PaddlePaddle-bot suggested changes Apr 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[KVCache] Support environment variable overrides for AttentionStore c…#7455

[KVCache] Support environment variable overrides for AttentionStore c…#7455
jackyYang6 wants to merge 1 commit intoPaddlePaddle:developfrom
jackyYang6:kvcache/as-config

jackyYang6 commented Apr 17, 2026 •

edited

Loading

Uh oh!

paddle-bot bot commented Apr 17, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov-commenter commented Apr 17, 2026 •

edited

Loading

Uh oh!

hong19860320 Apr 17, 2026

Uh oh!

jackyYang6 Apr 17, 2026

Uh oh!

hong19860320 left a comment

Uh oh!

This comment was marked as outdated.

Uh oh!

PaddlePaddle-bot left a comment

Uh oh!

PaddlePaddle-bot Apr 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jackyYang6 commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Usage or Command

Accuracy Tests

Checklist

Uh oh!

paddle-bot bot commented Apr 17, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov-commenter commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

hong19860320 Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

jackyYang6 Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

hong19860320 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

📋 Review 摘要

📝 PR 规范检查

问题

总体评价

Uh oh!

PaddlePaddle-bot Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jackyYang6 commented Apr 17, 2026 •

edited

Loading

codecov-commenter commented Apr 17, 2026 •

edited

Loading